Classifying Useful Motion Vectors for Efficient Frame Rate Up Conversion of MC-DCT Encoded Video Streams

Frame Rate Up Conversion (FRUC) is a technique that makes natural video playback possible by increasing the frame rates of video streams at the decoder side. Previous FRUC methods estimated the motions of the all blocks between nearby decoded frames to generate an interpolated frame, and hence, required a lot of computational resources. This paper proposes a new frame interpolation method that reuses the motion vectors (MV’s) that are generated by MC-DTC based encoder for primarily reducing the bitrate of video stream, but still useful for FRUC. In order to classify the MV to a useful one for FRUC, the visual similarities of two blocks that are located at the same position at the reference and current frames, the visual similarities of two blocks that are connected by the MV, and the visual similarity with the surroundings blocks, are measured. If these similarities exceed some thresholds, the MV will be directly reused for frame interpolation without additional block motion estimation in the proposed method. Additionally, if this decision is ambiguous, the pre-trained SVM (Support Vector Machine) is used to decide its usefulness. Experimental results using MC-DCT encoded videos with various motion energies show that the recall and precision of the proposed classifying method are about 0.72~0.99 and 0.83~0.99, respectively. Since the motion estimations for the blocks that have the MV’s which are classified as useful ones could be skipped in the frame interpolation process, the total time for FRUC can be reduced by 50-99% without a drastic video image quality loss.